Add fanout params for mongodb planning stage - #384

internal/app/options/connectorflags.go (1)

734-741: 💤 Low value

Optional: avoid the hardcoded 100 default.

The hidden flag duplicates the magic number 100, which can silently diverge from mongo.defaultDocumentDBSamplingFanoutLimit. Since the connector already applies that default in NewConn via setDefault, you could drop the flag's Value (or export/reuse the constant) so there is a single source of truth. Also note this flag is wired manually here while namespace-fanout flows through the proto/generated path — intentional given it's hidden, but worth keeping in mind for consistency.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/app/options/connectorflags.go` around lines 734 - 741, The hidden
flag in mongoFlagsCommandFlags currently hardcodes Value: 100 for
"documentdb-sampling-fanout"; instead either remove the Value so the connector's
NewConn/setDefault logic (mongo.defaultDocumentDBSamplingFanoutLimit) remains
the single source of truth, or reuse/export that constant for the flag's
default; update mongoFlagsCommandFlags to not duplicate the magic number
(reference: function mongoFlagsCommandFlags, flag name
"documentdb-sampling-fanout", NewConn and setDefault, and
mongo.defaultDocumentDBSamplingFanoutLimit).

connectors/mongo/conn.go (1)

249-249: Note the multiplicative fanout for DocumentDB.

eg.SetLimit(c.planningFanoutLimit()) bounds concurrent namespace planning, but each planning goroutine calls c.sampleIDs, which (for the DocumentDB flavor) opens its own errgroup bounded by documentDBSamplingFanout(). With both defaulting to 100, peak concurrency reaches ~100 × 100 = 10000 simultaneous $sample aggregations against the same cluster, which can exhaust the driver connection pool and pressure the server. This is still a large improvement over the previously unbounded fanout, so no change is required, but consider whether the effective product should be capped or documented.

Also applies to: 300-303

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@connectors/mongo/conn.go` at line 249, The current concurrency bounding uses
eg.SetLimit(c.planningFanoutLimit()) but each planner goroutine calls
c.sampleIDs which itself creates an inner errgroup bounded by
documentDBSamplingFanout(), producing a multiplicative fanout (eg.SetLimit ×
documentDBSamplingFanout) that can overwhelm the driver; update the logic so the
effective parallelism is capped or controlled—either reduce one of the limits,
enforce a global cap (e.g., compute min(c.planningFanoutLimit(),
maxGlobalSampling/concurrentSampleLimit) before calling eg.SetLimit), or add
coordination so c.sampleIDs uses a shared semaphore instead of its own
uncoordinated errgroup; locate and modify eg.SetLimit(...) and c.sampleIDs (and
documentDBSamplingFanout()) to implement the chosen cap or add documentation
explaining the multiplicative effect and recommended defaults.